The Idea of MM-Creole

The field of multimedia is composed of a variety of research areas. This diversity makes multimedia a special and interesting research field. However, the different vocabularies, methods, and cultures of the involved communities also introduce barriers that make it difficult to understand the field as a unified subject. Multimedia as a field seems to be divided according to different types of data. Therefore, many research groups for example work on either speech processing, computer vision, or video analysis. Related disciplines such as HCI and software engineering are often even disregarded as part of multimedia. The situation becomes even worse with the different disciplines having different vocabularies, methods, and community cultures. As a result, the subfields are organized into the same categories of data and different faculty members at universities often represent each of them individually. This makes it very difficult for students and other researchers to gain an understanding of the fields across these boundaries.

Today, the processing power of modern computers allows us to think about approaches that analyze a given media file "multimodaly", i.e. processing the audio, the images, and the dependencies between the images synergistically. Combined processing promises improved robustness in many situations and is closer to what humans do. The human brain takes into account not only patterns of illumination on the retina or periods of excitation in the cochlea, it also combines different sensory information and benefits from using as much context information as possible.

The book and this website are an attempt to make it easier to obtain a unified overview of the fields in order to be able to collaborate with people that work on different types of data by providing introductions to concepts, links to related projects and articles, event calendars etc... The idea of this site is to grow organically with contributions from different people in the different subfields.

If you have something to contribute, please feel free to add content. If you just read and enjoy this site, please spread the word!